Skip to content

manav1819/SecurityUpdateScrapper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🛡️ Security Advisory Monitor

A production-ready Python service that continuously monitors public security advisory feeds, detects new and silently updated advisories, and posts clean alerts to Discord.

Monitored Sources

Source Feed Type What It Tracks
CISA KEV JSON catalog Known Exploited Vulnerabilities actively used in the wild
Microsoft MSRC CVRF JSON API Monthly Patch Tuesday + out-of-band security updates
Cisco JSON listing / Atom RSS Cisco product security advisories

Architecture

┌──────────────┐     ┌────────────┐     ┌──────────┐     ┌─────────┐
│  Collectors   │────▶│  Analysis   │────▶│  SQLite   │────▶│ Discord │
│ (CISA, MSRC,  │     │  Engine     │     │  Storage  │     │   Bot   │
│  Cisco)       │     │ (new/update)│     │           │     │         │
└──────────────┘     └────────────┘     └──────────┘     └─────────┘
        │                                     │
        └── HTTP GET (requests) ──────────────┘
                                         advisories.db

Each poll cycle: fetch → analyse → store → alert.

Quick Start

1. Clone and install

git clone https://github.com/yourorg/secadvisory-monitor.git
cd secadvisory-monitor
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Create a Discord bot

  1. Go to Discord Developer Portal.
  2. Create a new application → Bot → copy the token.
  3. Under OAuth2 → URL Generator, select bot scope with permissions: Send Messages, Embed Links.
  4. Use the generated URL to invite the bot to your server.
  5. Create two text channels (e.g. #new-advisories and #updated-advisories).
  6. Enable Developer Mode in Discord (Settings → Advanced), then right-click each channel → Copy Channel ID.

3. Configure environment

cp .env.example .env
# Edit .env with your token and channel IDs

4. Run

# Test without Discord (see output in terminal)
python main.py --dry-run --once

# Run a single cycle with Discord alerts
python main.py --once

# Run continuously (polls every 3 hours)
python main.py

Project Structure

secadvisory-monitor/
├── main.py                  # Entry point and scheduler
├── config.py                # All settings (intervals, URLs, toggles)
├── requirements.txt
├── .env.example
│
├── collectors/              # One module per source
│   ├── base.py              # Advisory dataclass + BaseCollector ABC
│   ├── cisa_kev.py          # CISA Known Exploited Vulnerabilities
│   ├── msrc.py              # Microsoft Security Response Center
│   └── cisco.py             # Cisco Security Advisories
│
├── analysis/
│   └── engine.py            # Diff engine: new vs. updated detection
│
├── storage/
│   └── database.py          # SQLite schema + upsert logic
│
├── discord_bot/
│   └── bot.py               # Embed builder + AlertBot client
│
├── utils/
│   ├── http_client.py       # Rate-limited requests wrapper
│   └── hashing.py           # SHA-256 content hashing
│
└── logs/                    # Created automatically
    └── monitor.log

Configuration Reference

All settings live in config.py and can be overridden via environment variables:

Variable Default Description
DISCORD_BOT_TOKEN Required. Your Discord bot token.
DISCORD_CHANNEL_NEW Required. Channel ID for new advisory alerts.
DISCORD_CHANNEL_UPDATED Required. Channel ID for update alerts.
POLL_INTERVAL_SECONDS 10800 How often to poll (seconds). 3 hours default.
ENABLE_CISA_KEV true Toggle CISA KEV source.
ENABLE_MSRC true Toggle Microsoft MSRC source.
ENABLE_CISCO true Toggle Cisco source.

How Detection Works

New advisory: The (advisory_id, source) pair does not exist in the database yet.

Updated advisory: The (advisory_id, source) exists, but the SHA-256 content hash has changed. The hash is computed from: advisory ID, title, severity, affected product, published date, last modified date, and URL. If any of these fields change — even slightly — the update is detected.

Severity change: When an update is detected, the engine compares the old and new severity values. If they differ, the Discord embed includes a prominent "⚠️ Severity Changed" field.

Discord Message Examples

New advisory:

🆕 CVE-2025-21234 Windows Kernel Elevation of Privilege Vulnerability Source: 🪟 Microsoft MSRC | Severity: Critical Product: Windows 11, Windows Server 2025 (+3 more) Published: 2025-02-11

Updated advisory (with severity change):

🔄 cisco-sa-webui-csrf-yyyyy — Updated Cisco IOS XE Web UI Cross-Site Request Forgery Source: 🌐 Cisco | Severity: High ⚠️ Severity Changed: MediumHigh Last Modified: 2025-02-09

Running in Production

For long-running deployment, use a process manager:

# systemd (Linux)
# Create /etc/systemd/system/secadvisory-monitor.service
# [Service]
# ExecStart=/path/to/venv/bin/python /path/to/main.py
# Restart=always
# EnvironmentFile=/path/to/.env

# Or simply use screen/tmux
screen -S monitor
python main.py
# Ctrl-A D to detach

Extending with New Sources

  1. Create collectors/your_source.py inheriting from BaseCollector.
  2. Implement fetch() returning List[Advisory].
  3. Add an entry to SOURCES_ENABLED in config.py.
  4. Register it in get_enabled_collectors() in main.py.

License

MIT

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages